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ABSTRACT 

A series of experiaents to discover the optiaal 
learning situation were conducted using co»puter*assisted instruction 
based on a aatheBatical «odel. The instructional systeas were 
adaptive in that the sequence of instruction varied according to a 
given student's performance history, and the program was organized to 
■odify itself autoiatically as aore students completed their course 
and their records were fed back. Courses taught included coaputer 
prograaing for the junior-college level, reading for the first three 
grades of eleaentary school, and a foreign-language vocabulary 
prograa for colleges. It was concluded that an optiaal learning 
situation could be designed if the following eleaents were clearly 
specified: (1) the set of adaissible instruction actions; (2) the 
instructional objectives; (3) a aeasureaent scale that peraits costs 
to be assigned to each of the instructional actions and payoffs to 
the achieveaents of instructional objectives; (4) a aodel of the 
learning process. It was further concluded that a learning situation 
will result only if it is realized that the instructional theory aust 
be aodified to fit the learning process and that the theory of 
instruction will influence research on learning. (SK) 
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ADAPTIVE , INSTRUCTIONAL SYSTEMS : 

SOME ATTEMPTS TO OPTIMIZE THE LEARNING PROCESS^ 

Richard C. Atkinson 

Stanfo2^i University , . 

INTRODUCTION 

t 

One ca:.not help but question the feignificanee of psychology's con- 
tribution tc ithe development of effective instructional procedures. fOn 
the* one hand', psychology has been very influential in the field of 
education. In the last twenty- five years alircit eveiy major Innovation 
In educations-programmed textboolcs^x tehavloral objectives, un^|»ded^\ 
schools, individually iJrescribd^;''lijB computer liahagpd and 

assisted instruction, to^en economies, a^d talloWd te^tin^ to name a 
few-* can he traced tc psychology • In mAny cases these innovations have 
net-been due to psychologists primarily identified vith"^ education, but 
rather to laboratory scientists ^o«e research has su^^sted new 
approaches to instruction. Psychology can, be proud of that record of 
acr-omplishment. But upon close x* examination, it Is evident that these 
accomplishments are not as closely linked to psychological research as 
many might. believe. Psychology has suggested new approaches to education, 
but these suggestions have not led to sustained research progruus that 
have the promise of T>roduclng a 'truly effective thcoiy of Instruction. 
Rather, psychology seixas to provide the stiawliifi for innovation, but 



^o be published in Klahr, D. (Ed.)* Cognition and Instruction . Hillsdale 
Ifew Jenpey: Erlbaum Associates, 1975* i 



innovation that has not in turn led to a deeper understanding of the 

* i ^ 

Ipaztilng process . • 

Way has pi^yohology riot had a more -^libstaritia^ impact? Ttiere. are 
several reasons* The brightest and ablest young psychologists u^uilly 
are not attracted to educational research, and the research that h^s 
been done tends to be piecenieai, not pursuing problems in real jj^epth. 
Thi^ picture may cr.ange in tne near r>iture due to the limited lumber of 
Jobs fcr new P^..D. *5 and to society^s increasing emphasis on a pplied 
Tesearcri. The more serious problem, however, is that psycholc,/'^ sxs know 
a great deal abou^ the acquisition o^ individual facts and skil-^-s, but ' 
very little about hcv these combine to form a meaningful mentax stWcture 
Effectl^/e niethods for acquiring skills and facts are important, but the 
major problem the developrfient of knowledge structures that are more 
vr:dr„ ' ::r / uin -.f indiviiual facts. In order tv deal effectively with 
ed^-i.ational probleris, we need theories that tell us how knowledge is 
repre:ie:;ted in memory, how information is retrieved from that knowledge 
stracture , hew r.ew information is added to the structure, and how the 
3y:r*eni -:an expand that knowledge stracrare by self-generative processes, 
Th*" iev^lopment of such theories is under way, and increasingly work in 
cogniti'/e psychology is moving in that direction. The contributions of 
Anderson and Bo-^r (1973), Newell and Simon (1972), Rumelhart and Norman 
(1973), and Schank (197^) are examples of substantial efforts to develop 
compreFiensi'/e theories of cognition, and It is already evident that this 
work will ha^/e iinplications for education. Sucrh theories will not simply 
add another wrinkle tc educational researoih, but will lay the foundations 



for rese^.rch enccaapasbing a larger set of edueationally significant • 

problems than has been considered in the past* 

In thio^ papex: I want to review t^^e ongoing work in my laboratory 

that has implications for instmction* Some of that work represents 

/ 

attempts to deal with tne issue of complex knowledge stivctures, wh^»reas 
some is more restrictive dealing with the acquisition of spetific skills 
and facts* All of the work involves computer-based programs of instruc- 
tion used on a daily basis in schools and colleges. T^ese programs can 
best be described as adaptive instructional sys terns ♦ E|y that tern I 
meai; two things: (1) the Sequence of instructional actions taken by the 
program varies as a function of a given student's perfoimance histbry, 
and (2) the program is organized to modify itself automatically as more 
students cofflpLste the course and their response records identify defects 
in instractionkl strategies, 

[ Our work on adaptive instructional systems tJ^s three foci# One is 
the development of a course in computer programming for Junior college 
and college students; the second is a course for teaching reading in the 
first three grades of elementary school; and the third is a foreign- 
language vocaibulary program being used at the college level. This paper 
will review research on each of these projects. 

! \ 

i . / . _ • 

INSTRUCTION IN COMPUTER PROGRAMMING 



t > Our first/' efforts to teach comput^er prograraning involved the 
development jo/ a computer-assisted instruction (CAI) curriculum to teach 
the AID programming language ; this course has been used extensively in 
colleges and junior colleges as. an introduction to computer programming 



(B-ard, Lorton, Searic, & Atlcinson, 1973). Hcwewr, it is a linear, 
"frame •oriented" CAI program and Joe^i not orovide individualized instruc- 
tion during the* prcblem-solvdrig activity itself. After working tjircugh 
lesson aegraeata cr. iiyntax, expressions, etc., the student is asiiigned. a . : 
prpblem to solve in AID. He must then leave the instructional program, 
call up a separate AID interpreter, perform the required programming 
task, and return tc tne instructional program with an answer. As the" 
student writer hi-: program with AID, his only sources of assistance are 
the error messages prcvidfrd by the non- instructional interpi*4&.r. 

■'S. inadequacy of the AID course, especially for research purposes, 
15 its limited' ability to characterize individual students* knowledge 
Of specific skills, and its inability to relate students' skills to the * 
curriculuTi as anything more than a ratio cf problems correct to problems 
att/^r.ptrr The prcgram canrc make 'fine distinctions between a student's 
s!:r^ngths ani weaknf^ses, and cannot present instructional material 
Specifically appropriate to ^hat student beyond "hard«»r'I or "easier" 
lesscnc. In order to ^fxplore the effects of different currjculum selec- 
tion strategies in more detail, we developed another Introductoiy ^ 
programming course,' capabJ^e of rtpresenting both its subject matter and 
student performance more adeduately. The internal representation of / 
programming skills and their relationships to:' the curriculum is similai' 
in some ways tTj^the semantic networks used In the "generative" CM programs 
developed Ijy Carboneil and others (Carbonell, I97O, arid Collins, Carbonell, 
1 Wamock, 1973) i 



The BASIC Instructional Program ri ^ 

\ ' \^ \ ^ V 

An ifflportant featAire of a tutorial CAI program is to provide assis- 

tance.as the studeut attempts to solve a problemi The program must 
contain a representation of the subject matter that is complex enough 
to allow the program to generate appropr" assistance at any stage of 
the atudent^s solution attempt^* The BASIC Instructional Program (BIP) 
contains a representation of inf onnat ion appropriate to the teaching of 
computer programming tlxat allows the program both to provide help to 
the student and to perform a limited but adequate analysis of the cor- 
rectness of his program as a solution to the given problem. 

To the jjtudent seated at a teiminal BIP looks very mach like a 
typical timesharing BASIC operating system^ The BASIC interpreter, 
written especially for BIP, analyzes each program line after the student 
types It, and notifies the student of syntax errors- V/hen the student 
rune nis progra^, it is checked for structural iJ^egalities, and during 
runtime "execution*' errors are indicated » A file storage systfin, a 
calculator^ and utility commands are^^^avai labia. 

Beslding above the simculated oper^lng system Is the "tutor," or 
instructional program* It overlooks the 'entire student/BlP dialogue 
afid motivates the insti^ctional intex^ction*' In addition to selecting 

and presenting programming problems to the student, the IP identifies 

> ■ - ■ 

the student's problem areas, sTiggests simpler "subtasks," gives hints 

or model solutions when necessaty, offei^s debugging aids, and supplies 

Incidental instruction in the form of messages, interactive lessons, or 

manual references#\ . 



At BIP's cere is an .information network whose^nodes are concepts, 
skills, problems, 3Ub-pl:oblem$ , prt reqvisitec , BASIC cfcaanancis, remedial 
lessens, hints, and. aaiiual references. The i^|vcrk. ia used to charac- 
terize both the logical structure uf the ccurje and cur estimate ofn,he 
student's current state of Icnovledge; acre will be said abc at the network 

later. Figure i iliustratey the interactions of the par's of the ETP 

> ^' 
prt:'(3;ra.r.. 

c 

Trie curriculum is organized as a set of- prcg ramming problems whose 
text includes only the description of the problem, not lengthy descrip- 
ticnr. of prcgranaiing. structures or explanations of syntax. There is no 
fixed 'Ordering of the tasks; the decision to move from one .task, to 
ar.ctr-er in made on the basis of the information about the tagks (skills 
inv i-.'^d, prere.quisites , subtasks available) stored in BIP's network. 

A ■*'iie-*: prr jrc-r .^er, through the curriculum by writing, and running, 
a prcgran-. that solves the problem presented on hi:s terminal. Virtually 
no llmitatlo.ns are imposed on the as^cunt of time he spends, the nmber 
of lines ne writes, the number of Errors he is allowed- to make, the 
n^umber of times he chooses to execute the program, the changes he makes 
withih It, etc. The task on vtilch he is wortttng is stored on a stack- 
like structure, so that he may work on another task, for whatever reason, 
and return to the previous task automatically. The curriculiim stnicture 
can accommodate a wide variety of student aptitudes and skills. Most 
-of the .curriculum- re la ted options are designed w^th the lesr competent 
student in mir»*d. A more independent student may simply ignore the \ 
options. Thus, BIP giws students the opportunity to determine .their^^A^ 
own "chUlenge/l^ls" by makin*^ assistance «vailable byf^ot inevf table, 



HIP offers' the stnde^ cons Ide rat le flexibility In-maKing his own 

■ ' ■ - " - ■* 

task-r»"lftted decisions. He mgy asSt for hints and subtasks to. help him 

■^t started i,n ^giving tne given problt-ai, cr he ir.ay ponder the problem 
his cwn, uuing c :•;!>• the nar.ual fcr adiiticr.al i:; herniation. He oa/\ 
request a differerit task by rame, in the event that he wishes to wcrK 
cn.it isasedUtelo,', either cWletlns the n^w task cr not, as he chooses. 
Cn r.l3 return, 51? telli= hire the na/ne of the agaj.n-current task,, and 
ali:vj hi^-n t,t nave its text printed tc reminJ hia of tlie problem he . is 
tc ^cl'.'e. The^ £tuJK::.t say re^que^t the niodel joluticn for any task at 
any '.ir.e, but ,BIP viii net print**he ccdel for the -current task unless 
the student naa exhausted tne available hints and subtasks, Taketi to- 
getner, the curricul^om cpticns allov? for a vide range of student 

\ preferences yid behaviors. "^^'-^^^ 
BtP':^ Infcrriaticn Network 

Tajk jeie'ci'-r. . reniedfai ai^iiatancc. and problem area dVte*mination 
mquire 'that t.4 prcgraci na-.'e a flexifcir information store interrelating •< 
tarks, hints, sanual- l^ferencps, etc. This store has been built using 
tne a^scclatlve language' LEAP, a SAIL sub-language, in which set, list 

^a.nd ordered 'triple "data str^cftures are available (Feldman, l»ow, Swinehart 
h. Taylor, 19r72; Swinehart & Spn^ull, 15^1; VanLehn, 1973). Figure' 2 
presents a simplified relationship among a few programming concepta, 
specific observable skills that cnaracterize the acquisition of the \^ 
concepts, and pniigranj&lng ^pr^blems tr^at require the use of those skills. 
The network is cone trjcted' using the associative triple structure, an<^ 
in best -i^^f.orifc-f d in terms 'vf vartr;ijr. type;; of nodes: 



TASKS All curriculxin elements exist as task nodes in the* networit. 

. Th«»y are "linked to each tither as subtaskiJ, prerequisite 
..tasks, or '*iHust follow" tasks. * 

amLS T^i' skill node? are intemediailes between tMe %onoept nodes 

and t^e task Troties (Figure 2) / Skills are vejy specific,' 
'e.g., *ccrtcate:iatln^ iitrixi^ variableii" cr "incrementing a 
cottiiter variable.** ^ evaluating success on the individual 
skills, the program estimates canprtence levels in the con- 
(*pt areac. In the network, skills are related to the tasks 
th«t require them and tc the concepts that embody them. 

CpNCEPTr„ Tr.e principal concept are&s cove red. by Bl.f* are the following: 
ir.tentctive prcgrans variables and literals; expressions ; 
ir.put and output; prc^gram control - branching;' rfepetition - . 
* loops; debugging; subroutines; and arrays. 

CPER>i'?'H£ Each SASIC operation (PRINT', LET, ...) 1^ a node in the 
network. i..The operations are linked to the tasks in tvo 
-ways; either as elements that must be used in the solution 
* . cf the problem, or as thoge that mus* not be used in the 



Tr:e nint nodes are linked te the tasks for which they may 
be helpful. Each time a new skiXl, concept or BASIC pperator 
is intrc^luced_^ Ih^^^ is an extra hint 'that .gives a suitable 
.•nanuai reference. 

All di = ccverabit-- syntax, ,6* rut tural, and execution errors 
exist as nodes in the network, linked to the relevant "help" 
messages, manxtai references and remedia lessons. 



is ^ 



Clearly, I'n sop^ ,;;.a5es , a hierarchy among skills or problems 

i 

inipiiclt; n:ore frequefitifly, hcw«*wr^4uch a relfttionshlp cannot be assvuned. 
By L^posing only a very loose hierarchy (e-g.^ requiring that el^ students 
begin the course wltn the zasie probleja), it is possible to select currtc- 

'4 i 

uium and provide assistance on the b^sis of a student's demoxiitrated 
c<^3n!petence iewl on spfMflc skills, rather th^an on the basis of a '^re- 
dete mined, ncnindividualized, sequ<;nce of problems. Students who acqui^ie 
competence in skills in. seme manner other than that d|ssuffied"'^>liji^Ject-, 

matter experts to be standai-d should benefit most from this potential 

■ ' * ■ - 

for individualization. 



"^^^^J""^^^ completion of a taskj the student is given a "post tasK inter- 

. ■ \ ' • . . • ■■■■■ 

vlt:w** in which BIP presents the model solution stored for that prolilem* 

The studentHs encouraged to regard, the" mtodel onXy ore of many possible 
sclution:^* ETP asks? the student vhether he has solved the problem, then 
aisks (for each of tW skills associated vith the task) whether i^p needs 
rscre pjBCtice involving that skills In addition tQ.the information 
gained f rom this' student seXf*'?nalysis, BIP also stores the result of a 
compariiicr: between the student's program and the model solutioii, ^aed 
on the output of both programs when run on a set of te^t d^ita. The 
::tudent*s responses to the interview and the results o^f tlife program 
Cv::;mparison are used , in future BIP-gene rated curriculum decisions. BIP 
irfoms the student that he h.as ccmpleted tire task ^ and either allows 

nlrn to select his next ta^k by nanje tf-ncci an off-line printed list of 

" f, 

na*nes and problem teicts), or selects it for him* 

nr. example of the role of the Infcxmation Network in.BIP*^ tutorial 

capabilities is the BlP-g^^ne rated curriculum decisions mentioned above. 

^ ^^"^^rlng the ^tudenr»s ovp evaluation of his skills, and by comparing 

ni.- solution attempts to the stored models, HEP can be said to "learn*' 

about each student as an individual who has attained a certain level of 

competence In the skills associated with each task. For example^ BIP 

* 

might rmve recorded the fact that a given student had demonstrated com- 
peten::e (and confidence) in the skill of assigning a literal value to a 
variable (e.g., H/= 1), but had failed to master the skill of incrementing 



a counter variable (e.g., N ^ lUl). HTP can then search the network to 
locate the ?!kilI^Vtha?t are appropriate to each stxident*s abilities and 
present tasks thj^t thcorpora*^^ those ski-ils. The network provides the^ • 

• 12 

'/ 

-.15 



tjase-lrroiii which BIP can generate iecisiono that take into account botn 
the subject matter and the student, behaving soniewhat like a human tutor 
In pVesent-ing ir^terial taat either correct^^ specific weaknesses or 
challenges ani extends particular strengths, pr^ ceediug into a^, yet un- ^ 
encountereci area^;. * 

> Ttie ElP program has been running successfully with both Junior 
college and university students. However, the progr^ is still very 
auch in an experimental stage. From a psychological viewpoint, the 

\ 

N 

principal researcn issues ieai vtth (1) prcoedures for v:)btalning on-line 
estiniates c? student abilities as represented in the informatton netwoilc 
and (2) alternative irtetncds for using the current estimates in the in* 
r^rsnation network to make instractional decisions. Neither of t^iesj^-- 

is re-.tricted to this particular course, and a major goal. Iri the 
:iev*"^l-prr:^n^ *" f BIP is to provide an In.strudtional model suitable to a 
varievy of iiffervnt subject areas. Tvo topics mast be discussed in 
relation to this goal: the nature of appropriate-s^itject areas and the 
g-r.eral Gr.aracteriitl'-,s of tte BIP-like structure that maRes It paipticu- 
l&rly u:>eful in teaching such subjects. 

A rabj'ect well-suited to this approacn generally fita the following 
iescription: it has clearly definable, deitonstrable skills-, w^ose 
rela^lcn^hips are well-known; the real content of the -subject matter is ■ 
of a problem- solving, ratner thar. a fact-acquiring, nature i the problems 
presented to the student involve overlapping sets of skills ; and a 
student's solution to a given problem can be Judged as adequate or in- 
adequate with J ome degree of confidence. The BASIC language, as taugnt 
by HIP, is one such subject,' but the range of appropriate curriculums 

IG ■ • 



goes veil beyond t!ie area of computer science* For example, elementary 
statistics co^.d be taiight by a stmlX&r approach, as could algebra, 

navigation, accciwiting, or organic chemistrj-^ All these subject areas 
involve the nianipulktion of information by tiie i^tudent toward a laiown 
goal, all invcly^ processes that can be carried out or simulated by a 
computer, And all are based on a body of skills w lose acquisition by the 
student can be measured with ^ acceptable degree of accuracy* 

Because they require the deAfeloiKLent of problem-solving skills, i 
■rather than rrjfe n;emori2ation of facts, these subject areas are frequently 
litficult to ciaster and difficult to tutor, especially using standard ^ 
CAT techniques. One limitation of sucn standard Aechniques is their 
iependence on a **right*' answer to a giyen question or problem^ which 
nn-cl'Uvies active student participation in a problem- solving process 
c onsisting of many st?eps , none of which can be evaluated as correct or 

rre^t except within the context of the solution aS a whole^ In 
addition, standajrJ CAI techniques usually consist^ of an instructional 

c 

facility alone — a mechanlsn: by which information is presented and 
I'^^tp/.inc^es are Judged* Tni^^ facility can be linked to a true problem- 
evolving facility that allows the student to proceed through the steps 
to a solution, but the link does not allow the transfer of infoimation 
betvi?en the Instra-stional and thp problem-solving^ portions of the program. 
The complete integration of the two parts is a k^y feature of EIP^ makilhg 
It appropriate to instmction In subjects areas\t^iat have been inadeqimtely 
treated In CAI • '^^'^^'^ . ^ - * / 

The most general characteristics of the*''^etv/ork" structure Include 

/■ 

a representation of the curzlc'-ilum In terras ot the specific sltllls 

1 
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require^ in Its ,masteiy an^ a representation of the student's current 

* * 

levels of coflBpetence in each of the skills he has been required to use* 
Indl^dual record*keepiag relates each student's progress to the cur- 
riculum at all times ^ and any number uf schemes may be used tc apply 
that relationship to the selection of tasks or the presentation of 
additional information, hints, advice, etc* 

An important element of cur netwotk structure is the absence of an 
established path -through the currlc,ul\«n, i)rovidlng the built-in flexi- 
bility (lixe tMat of a human tutor) to respond to individual students* 
Strengths and weaknesses as each student worits with the course. This '* 
can cnly be accomplished through a cariful analysis and precise specifi- 
caticr. of the skills , inherent in subject matter, the construction of 
a thorough curriculum providing in-depth experience with all the skills, 
and*a structure of associations among elements of the curriculum that 

i 

allows* for the Implementation of 'various instructional strategies.'' 
Instnictlonal flexibility Is complemented by research flexibility in 
sucn- a stracture, because the nature of the associations can be modified 
for dl»fferent experimental purposes. Once the elements of the network 
have been established, it is easy, for example, to change the prereq- 
uisite relaltonship between two problems, or to specify a higher level 
of competence in a given skill as a criterion Masuite. 

The cotislderable complexity involved in programming this kind of 
- flexible structure inrposes a certaiii limitation. Standard CM "author 
languttg.es" arc not appropriate to this netwoA approach, and constructing 
a CAI course on HTP's pattern i^ ^not a task to be undertaken by the 
educator (or researcher) who has. no programming support,^^The usefulness 
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of author languages Is their simplicity ^,jwhich* allows subject-matter 



author languages provicie for alternative parhs through a curriculum, for 
altematiw answer-matching schemes and so forth; considerable complexity 
is certainly possible. However^ the limits, once reached, are real, and 
the author simply cannot expand the ^iiophistication of his course beyond 
these limits. 

The programming support required by the network approach^ on the 

ether hand, implies (l).the use of a general, powerful language allowing 

/ 

access to all the capabilities of the computer itself, and (2) a pro- 
gramming group with the training and experience td make full use of the^ 
ziacnine* It has been our experience that the flexibility of a general 
purpose language, while expensive in a number of ways, is ^worth the costs 
by virtue of the much greater freedom it allows in the construction of 



tne curriculum and the implementation of experimental conditions. For 
a more complete descrlptKn of BIP and a review of our plans for further 



aimed at a total curriculum that would be virtually independent of ^he 
classroom teacher (Atkinson, I968). These early efforts proved reason- 
ably successful, but it soon became apparent th^t the cost of such, a 
program would .b© prohibitive if applied op a large-scale basis. Further^ 
it was demonstrated that some aspects of instruction could be done very 
effectively using a compater, but that there were/ other tasks for which. 




re3earch see Barr, Beard, and Atkinson (197^)* 



INSTRUCTION IN INITIAL READING (GRADES 1-3) 



^ Our firot efforts to teach reading under computer control were 



/ 



16 



the computer did not have any advantages over classroom teaching. Thus, 
during the last four years, our orientation has cimnged and the goal now 
is "to develop low-cost CAI that supplement = classroom teaching and ccn- 
. c<Bntrateri on those tasks in which IrKilvidualiaatlo:; i;. critically i-nportant, 
A ot^idenr terainal in the current p;:ogratn consists only cf a Model-33 
telet^evnter witr. an a-odic head:3et. Tnere is no graphic or photographic 
capability at the student teminal as there was in our first system^ and 
the c^-^racter set cf the teletypewriter Includes o^ly uppercase letters. 
bn tr.e v-h.pr nand, the audio system extremely flexible and provides 
'/Irtuallv Instantaneous access to any one of 6,000 recdi^ded words and 

■ \ 

messages. ^ 
Reaiing dirrlculum 

Pradlng Inst/'iction can be divided into two area^ which have been 
r^ *"*' rr*:^i t : a:^>Tecoding'' a-d "cornnunicatlcn/' Decoding is the rapid, 
if not autcmatic, association of phonemes or phoneme groups with their 
n?;:ipecti^^ graphic representations, » Communication involves reading for 

meaiiirig, ae3tri**^c enjoyment, emphasis, and the like. Our CAT program, 
nr^'vdie^ instraction in both types of tasks, but focuses primarily on 
iecoding. The prcgranr is divided into eight parts or strands. As 
indicated in Figure 3, entry into a strand is determined by the student's.^- 
level of acnievement in the ^ther strands. Instruction begins in Strand 
0, which teacfte-3 the sltills rJ^quired ^'interact with the^ program. Entry 

' into the other strands is dep^kdent cm thfc stiodeat's pcrfoiroance In 



earlier 3*randJ. For example, the letter identification strand starts 
with a .subset of letters used tn the earliest sight wordf.. When a 
student reaches a point in th* letter idj*ntification strand where he ilps 
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Figure 3, Schematip presentation of the strand structure. (Entiy 
into each strand depends on a student's peTfoimance in 
earlier strands^ 'The vertical dotted linea represent 
vinaxlaaX rate contours which control the Student's progress 
In each strand relative to the pther strands.) 



la 



exhibitjBd mastery ever the letters used Iri the first words of the sigtit- 

word^trtmd, he entevs that atrand* Similarly, entry i'litc the spelling 

pattern strand and The phonics straiid 1;: controlled- by the student's 

placemesit in the sight -word' strand-. Or. any given day, a tJtudent may bo 

seeing eltercises drawn from' as many as five strands. ,-,The dotted vertical ^ 

lines in Figure 3 represent ^.'^naximal rate cent ours;" w.h^ch control the 

student»-s progress In each strand relative to hi^ pro^ss in .other 

strands. The raifici.ale underlying these contcur's^is that leanXng par- 

« - f . ' \ 

ticular r:aterial,in one ''strand facilitates learning in another strand; 
thus, the contours are conscyiucted so that the student learns specific 
itens from o»e strand in conJ\incl:icn with specific items froft other 
5 1 ran 3s. 

The CAI piNDgram is highly Individualized so that a trace througn 

:urTiculUT. Is unique for each student, Cnir problem is to specify 

hew a giver; subject's response history should be used to make instruc- 

tional decisions. The approach that have adopted is to develop 

n;a'r.fernar,ical models for^the asq-^isition of the various sltllls in the 

rarnculum, and then use these^*!rcdels to specify optical sequencing 

:^cheme^« 'Basically ^ this approat^il is wha^ has come to be known in the 

engineering literatu?:^ as "optimal control 'theory or^ mo|jp simply, 

''control theory/' Precisely tne same problems are posed in the area of 

instruction, exeept that the ^system to be coptrolled is the h\«nan learner 

ilither thafi a machine . or group of Industries^. If a learning model can 

be specified, then methods of control theory can be used to deri'/e 

I 

optimal instructional strategics • 



Sane of the opt^jnTzaticn^^rc^^rdure MlllA>t rtvlewea later , but in^ ^ 

off thi 



lorter for jtliU'. reader to have sbae of hof the CAI program opa^tes^ 

' let me fi^t describe a few of the simpler exercises used in Stran 
^. J ITI| And IV* ^^'f^d 11 provides for^ the development of a slght-vord ^ 

vocabulary. Vocabulaiy Items are presented in five exercise formats; 

cihly the copy ijxercise and the recognition exercise will be described 

■a. -I ' ^ 

here. The top jpanel^of Table 1 illustrates the copy exercise^ and ^ the 
iower panel ^illustrates the recognition exercise. Note that when a 
Student maKes an error^ the system responds with an aud^o message ,and 
prints out the correct response. In earlier versions oil the program, 
the student was required to xopy the correct response following an error. 

'I 

Experiments demonstrated that the bvertf correction procedure was not 
particularly effective; simply^ displaying the correct wo^ following an 
error provided more useful feedback. 

Strand III offers practice with spelling pattems'^arid emphasizes 

. i 

the regular grapheme -phoneme correspondences that exist lin English. 

■ ' ■ i 

Table's illustrates exercises from this strand* For the Exercise in 
the top panel of Table 2) the student is presented with three words , 
involving the same spelling pattern and is required tto select the cor- 
rect one based on its initial letters. Once the stWent; has Reamed to 
use the initial letter or letter sequence to distinguish between words, 
he moves to the recall exercise illustrated in the bottom panel of 
Table 2. Here he workf vittiyB ^roup of words, all Invo'^^tae the s^t 
spelling pattern* On each trial the audio system regtieetp a* wojcd- t^t 
requires adding, an initial consonant or consonant clust^fr^ to ih^ spelling 
pattern mastered in the ^^i^ceding exercise. Whenever ajsitudentXmalies a 
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.correct response, a *'+" sign is printed cv. tne teletypewriter. In 

addition, every so often the prcgraoVili' give an auiit^J' te^'dback message; 

V • , - „ — 

these messages vary from simple ones liice "great," "that • 8 fabulous," 

"you're doing brilliantly," tc .vcme' that na%'e cheering, clapping, cr* 

belle ringing in the background. Thes^ messages are not generated at 

randcc, but. ;ieper"!ci on the student's perfomance on that particular day. 

. WVien the student has mastered a specified number of words In the ^ 

sig^t*vcrd strand, he begins exercises In the phonics stran^j^ihis stranci 

concent rates on initial and- final consonants and consonant cltistere In 

ccmbinatlon with rsedial vowels* As in most linguistically oriented 

cu^^cula,^. iitudents are not required to rehearse or identify cb^isonant 

sounds ir. iaolation. The emphasiis is on patterns of vovels and con- 

:cr.ar.t3 that bear regular correspondences to phonemes* The phonic strand 

Is tne xcst complicated one of the group and involves eight exercise 
f 

r r!nat3; two cf tne formats? will be described here. The upper panel of 
Table 3 illustrates an exercise in which the student Is required to 
lier.tif^' th^ graphic representarlon cf phonemes occurrirgat the end of 
wori£. Each trial begins with an audio presentation of a word that 
includes the phonemes, and the student is asked to identify the graphic 
rep re sen tar ion. After mastering this exercise, he is transferred to 
the exercise illustrated in the bottom panel of Table 3. The sane 
phonemes are presented, but now the student is required to construct 
words by adding appropriate consonants. 
Optimal Sequences for Individual Students 

This has been a brief overview of sonje of the exercises used in the 
curriculxac; a more detailed account of the program can be found In 
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Atkinson, Fletcher, Lindsay, Campbell, ni.d.Barr (l«r/3). ^-c Key to trxe 



curAculUE Is the oi^tlalzatteiN ^cheoed control the sequencing of 

the exercliie:i; tlj^stt sch^'sird tan be classifie^l at thr«?e level:*. One . 
lewl ip.v...lv<rr. ieotsiun lading wltnin eacr. stmr.sl. The problec Iti to^ 
declie wr.ich itemii to present for stuOy, rfhlch exercise formats to pre- 
sent thes ir^ and when to schedule revlev, A coffiplete response ^history 
exists for each stiuierit, »r.d this history is used tc make trial-by- trial 
de::ii;lor.6 regarair^ what tc present next-. The second lewl of optlmlza- 
tier, lealr. wit', decisions about all»;aation of instructional time ainJong 
^nrar. lr for a giver. &t>jemi.-*~Ji At tne er^d of an indtructional session, 

ir.e ;- M4er.t will ^iave ^ache?d a certain point in each jtrend ana a 

s ■ • 

iecl ^l :r; niu.;t be maie *Bb.ut trie tl»e tc be allocated to each strand in 

..xr -estivr,. The tr.ird level of optliLizatlon deals with the die - 
•'rl: -t;,:r, f L:. tract! onal t4^- azaong ^tudent<i. The question Kere is 
liA.ctitr- .Nr.pur«^r uacng .ituieiiii." acrJ-e^- ir.st|inicticnal ^ 

rjectlv-ts tnat are defiriei not/^t the Individual stud«^t but for the 
cla4ir a 9 wr.:i**. In ^ase global senile, fheue three levels of optiiKlza- 
L .^.',,aiu be integrated lr:tc a unified programs. Hcvever, we have been* 
.ja'lr-i'ted tc wcrk with each sepamt^ly, r ping that later they can be 
\->Z'^xrs^TH*x \ ;*:.to a .Urig^ pacKage. 

^/ptlai station *rt.thiri a dtrend (what has Jbeen called Level. 1) car^ be 

... /' 

lUuiT^rate'i "^^g *-he sight-w-^rJ strstid. The strand compilses a list 

f 9t>DUt 1,000 w r^i; J th*e wi.ric ss:^- rdered in tertii£ of their frequency 
In thp £:tiider,t*5 vf ca^ulary, and wcrdi; at the beginning of the list have 
nl^nly reflruiar grapht^ae -pTvCriSaicr cTrn?£p<.nder . At^ny point In time 
a stud<(»n.t will working "on a IKhi^^ed ptx;! of words from the master 



Itst; the size of this working pool depends on the sttident's ability 
level and is*iusually between 5 and 10 words. When one of these words Is 
mastered, It i deleted from tne pool and rt-placed by tne next word on 
the list or by a vori due f- r revie\f^ Figurv ♦ pre-^ents a flow chart 
for I'he strand* Each word in the working pool is in one of five possible 
Instructional statet>. A trial invulveri sampling a word from the working 
peel and prei^enting it in an appropriate exerc:.:3e fcmat* The student 
is pr^':ested < n a vora rnr firi::t rVw time;;? it is presented to eliminate 
wcrd^* already kr^n. It he know$ the word, he will pass the pretest and 
tr.e w.ri will b4r lr:pped frum the working pool. If the student does not 
pa::^ tne prt?teiit» he first istudies the word uSing the recognition exercise 
If review is required* he- studies the word again in what is designated 
rij^-ire as Exercii^es k and 5^ 

A.;- ir:ilcat<^i in rlg*^re a given word passes from oneNstate to the 
i' r^:5i:.:.v.. cr:t-ri':., Ai.d *rA6 prejent£ the crux of the opti- 



rl^aii ,:; pr jblez.^/Iicr. 16 to defir.e ai appropriate criteilon for each 
exercii^e. Thl- nsLii been done ujing ;>imple mathematical models to describe 
* fTt "Hc/^i^i-'i pr^jce^^ frr each 'r-xercije and the transfer functions that 

noid betweer. exe raises (Atkinson & PauL^on^ I'^d)* These models are 
;:rin:ple MarKov prc^esses tnat pr* .%i.de reasonably accurate accounts of 
performance our ta^K^., parameter r^i of the models are defined -^s 
f-jLncticns of two factors: (l) the ability of the particular student, 
and {c'.) the difficulty of the particular word. An estimate of the 
student^s ability is obtained by analyzing hi6 response record on. all 
previ/u^; word.;, and an e:;ti:nate of a word'j difliculty is obtained by 
analyzing performance on that particular word for all students on the 



tmmtmm 




Figure ii, 



partial fl^ chart for Strand II (slght-vord recognition). 
The various decisions represented in the bottom part of ' 
the chart are based on fairly ccanpllcated coaq?ut«tlon$ that 
iD'^e use of the 8"ftident*3 response hiapory. the same 
recognition exercise is used in both state and S^. 
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program. The student, records are continually updated by the cWputtr ;;' 
and are used to compute a maximum likelihood estiiikate of each stu^f^ 
ability factor and t?ach''vcr:l ' g difficulty factor. Given a vellvde^neid 
aiadel and c^tiniatec of its parar.eterL' , can ui^e the methods of control 
theory define an optimal criterion for each exe'rci-Se* The criterion 
will v^ry dep^'-nding on thewfif ficulty of the item, the student's ability 
le\^l, and the precise sequence of correct and incorrect responses ade 
by the 'student to the item. It Is important to.realiaie that the optlmi- 
zation scheme is not a simple branching program bjased on the student's 
ia. t response, but depen«43 in a complicated way on his complete response 
History • - 

Optimization between strands (what has been called Level II) was 
T.entioned earlier in tne description of maximum- rate contours. In' some 
rei:pects tni^T cptimizatlon program is the most intere::ting of the groups 
bu:. It jaz-i'vi be c'xpiaine^i witn'^jui g*-iilg into conslder^^ble mathematical^ 
detail. In essence, a learning model ii^: c(eveloped that specifies the 
learning rate on each strana as a Tonctton of the aibotint of material that 
na.: been mastered in each of the other strands. UsiSg mathematical 
methods of control theory, an op.timal instructional strategy is determine 

based on the model. This strategy defines a H:losed-loop feedback con* 

/ 

troH^r that specifies daily instructional allocations for eacfh ^rand 

J . ' 

based on the best current estimate of how much tHe ftudent has mastered 

in each strand. An account of the theoretical rationale for the program 

is presented in Chant and Atkinson (1973)« m 
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Htert let lis consider an example of optiiatzation at what has been 
called Level III. The effectiveness of the CAI program can be increased 
by optimally allocating Instractional tjme among titudents. Suppose that . 
a school has budgeted a fixed amount of time for CAI and nnist decide how 
to allocate that time among a class of first-grade students. For this 
example, maximiting the effectiveness of the CAI program will be inters 
preted as meaning tHiat we want to maximize the class perfomancc on a 
standardized reading test administered at the end of the first grade. 

On the basis of prior studies, the following equation has been 
developed to predict performance on a standardized reading test as a 
function of the time s student spends on the CAI sy8,tem: 

P{t;i) = A(i) - B(i)exp(-tC(l)l . 
The equation predicts Student t's performance on a standardized test as 
a i^anctlon of the tine, t, spent on the CAI system during the school 
year. The parameters A(i', 3(i), and C(i) characterize Student i, and 
vary from one student to another. Th«se parai^ters can be eatlmtted 
from scores on reading readiness tests and from the student's perfcnnance 
during his first hour of CAI. After estimates/- of these parameters have 
been made, the above equation can be used to predict ertd-of -year, test 
scores as a function of the CAI time allocatwd to that stxidentt 

Let us suppose that a school has budgeted a fixed amount of time T 
on !thc CAI %steffl for a first-grade class of N students; further, suppose 
that students have had reading readir*ss tests and a preliminary run on 
' CAI system so that estimates of the parameters A, B, and C have been 

made for eac>- student- The problem then Is to allocate time T among the 

■U 
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N students so aa to optimize learning* 'In order to do this, it is first 

necessary to have a model of the learning proces • Although the above 

eqviation does net offer a very detailed account of learning, it suffices 

as a model for purposes of this problem. Thit' is an import^ant point to 

keep in mind; the nature of the specific optimization problem dete mines 

the leyelvof complexity that needs tc be represented in the learning 

model. For aome optimization problems, the model must provide a rela,^ 

ti'/elj' detailed account of learning to specify a viable strategy, but 

for other problems a simple descriptive equation may suffice* 

In addition to a model of the learning process, we must also specify 

an instructional objective. Only three possible objectives will be 

ccnsidered here: 

I. Maximize the mean value of P ov^^r the class of students. 

II* Minimize the variance of P over the class of students. 

Maximize tne mean value of P under the constraint that the 
resulting variance cf P is less than or equal to the 
variance that would be obtained If no CAI were administered. 



M for t^J^^I 



Object t'/*^ I maximizes the gain for trie class as a whole; Objective IT 
reduces Jlffei:ences among students ay making the class as homogeneous 
as possible; and Cbjectlve III attempts to maximize th^ class perfomatic*^ 
wnile insuring that differences among students are not amplified by CAI. 
If we select Objective I as the Instziictlcmal ob^Jectlve, then the probXenL 
of deriving an optimal strategy reduces to maximizing the function.;^ 

f(t(l),t(2),...,t(N)j\ 2_ {A<l)-B(l)expl-t(l)C(l))) 

i ■ . ■ ■ , 

t(i) + t(2> + ... + t(N) - T 
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where t(l) Is the time allocated to Student 1. This maximliatton can be 
done using the methods of dynamic programming. To Illustrate tl» apprtmch, 
coaptations were made for a first-grade class for which the parameters 
A, B, and C tiad beer, estimated for each student, iinploying these estl- 

.3"' . ■ • 

mates, computations vepe carried out to determine the time allocations 
that maximized the above- equation. For the optimal policy, the predicted 
mean performance ^evel of the class on the end-of-ycar tests was Ikf 
hightr than a policy that allocated time equally among student a (i.e., • 
an equal- time policy where t(l) = T/N for all 1). This gain represents 
a substantial improvement; the drawback is that the class variance is 
roughly 15* greater than the variance for the clasE using an equal- time 
policy. This means that if we are only interested in raising the class 
aw rage, we will have to give the, rapid learners substantially more time 
or *he CAI system and let them progress far beyond the slow learners. 

Although a time allocation that conplles with Objective I does 
increase overall class perfomance, other objectives need to be considered. 
For ccaaparison, time allocations also were coo^mted for Objectives II and 
III. Table h presents- the predicted gain in average class performance 
as a percentage of the mean valw for the equal-tine policy. Objectl^ve 
II yielded a negative gain in the mean; and so it shqpild, since Its goal 
was to minimize variability, which Is accompUahed by reductog the time 
allocations for rapid learners and giving more attention to the slower 
ones. The reduction In variability f or Ohjectiw II Is 12** Objective 
III. which strikes a balance between Objective I and Objective II, yields 
an 8* gain in mean performance yet reduces variability by 6*. 
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Table k 



Pi«aieted Percent 0«ln In the Msan of P and in the 
Varlanos of p Vten Oot^axva vtth tb« Nean and 
y«il«noe of tb« Iqual-TlM Policy 





"* Inatzuetiooal objective 


\ 


I 


II in 


gain in mean of P 




-15 a 


gain in variance of P 


15 


-12 -6 
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In view of these results, Objectiw III vould be preferred Ijy moBt 
educators and laynen. ' It offers a substantial Increase in average per- 
fQimanca while maintaining a low level of variability. These cooputatlons 
make it clear that the selection of an Instructional objective shoulti not 
be done in Isolation but should involve a comparative analysis of several 

ft " 

objectives, taking into account more than one diaiension of perfomance. 
E/en If the principal goal is to maximize the class average, it is in- 
appropriate in most educational situations to select Objective I over 

III if it U only slightly better for the class arcrage, while pentitting 

2 

variability to mushroom. 
Effectiveness of the, Reading Program 

Several evaluation studies of the reading program have been con- 
ducted In the last few yeajra. Rather than review these here, I would 
prefer to describe one in some detail (Fletcher.* Atkinaon, 1972). In 
this particular study, 50 pairs of kindergarten students were matched on 
a nunber of variables, Including sex and readiness scores. At the start 
cf tnp first grade, one member of each pair was assigned to tha^xperl- 
mental group ar; i the "other to the control group. Students in the 
experimental group received CAI, but only during the first grade; students 
in the control greup received no CAI. The CAI lasted approximately 15 

minutes per day;-^ during this period the control group 0tudl«(d reading 

ft ' ■ 

if • 

^For a more detailed dlacussion of some of the issues involved In ^ 
selecting objective ^fUncti<^ see Jamison, Fletcher, 3)4>pes, and 

Atkinson (1975). 

^In this otudy no attempt was made to allocate time optj^ally among 
students in the cxpexl^ntal group; rather, an equal-tike poilcy was 
eag>loyed. / » ■ 



In the classroom. Except for this 15*in.lnute pertod, the school day for 

r " 

the CAI grcfup vaa like that of the control grou^. 3tandarditcd tests 

f 

were adainis tared at end of the first grade afid again at the end of 

* 

the second grade. All the tests showed roughly the e^irae pattern xf 
results; to suaniarlze the findings, only daC* from the California Cooper- 
ative Priinary Reading Test will be described. At the end of the first . 
grade, the experiment ad group shoved a 5»05»mcnth gain over the control 
group. The group!;, wnen tested a year later (with nc intervening CAI 
treatsient), Lncvei a difference of ^^.90 months; Thus, the Initial dlf- 
rerence ob:>er'/ed following one year of CAI was aiaintained, although not 
ainplifiei, during the second year when no CAI vas administered to either 
gr^up. ^ 

No Jefiniti'/e conclusions can be drawn from evaluation studies of 
*:r.i/ ;rt abcut the specific contributicns of CAI versus other aspects 

*-.e ji* uatlor.- Lbvl^uily "he c-^rrlcul^um mateilalj used in the CAI 
prograflT; ar^ import ar.T\, d6 well 'a^ other factors. ^ To do the type of study 
tr:at would isolate the ixpcrtant variables is too laigc an undertaking 
^ w-rtr:wriil- at this Juncture in the development of the readlrig pro- 
gram. Tnus, to isome ex-»^ent it Is a matter of Judgment In deciding which 
•/ariabif^i a:-.vrurit for *>nrr differences observed xn the above study. In 
my view^ individualizing instruction la tk'ie Key factor In successfully 
teaching reading* This does not mean th^t all phases of instruction 
snould be individualized, but certain skills' can be mastered only if 
instruction Is sensitive to the student^s particular difficulties « ,A 
reading teac^^er Interacting or. a one-^o-one basis with a student may be 
tuore ef/ective tnan our CAI program. Howi^ver, when worKlng with a'group^ 

3^ 



ct children (even as few as four or five), It is unlikely that she can 
match tlie computer's eff*ctlvenec.i; ii/niaking iifstruttionail ieclstons 
over an exiended perlcnl of time. 

SECOND- LM'GUiCE VOCABIL^m l^JMlUG 

m 

In this section, research on CAT prograas for second- language vocab- 
ulary learning will be discussed. As noted elcevhe re in this paper^ the' 

\ 

pclii2±pal goal cf our research o*n compute rli«»d instruction has been to 

ft ■ 

develop adaptive teaching prccedures—procedures that make monent-by- 

*■ " . ■ . • 

aomerin Jeeiciv-niE about which instructional action should be taken next 

basei on toe student's unique response history. To help guide the 

theoretical aspect of this work, 3c*ae years ago ve ^initfated a series 

It' experisents CT. the very restricted but well-defined problem of 

.ptl~dzir.g the teaching of a foreign-language vocabulary. This is an 

:irt'\ w:.erc .T.at. :.f;.T.a!:lcal z/^delc przvidG ar. accurate description cf 

learr.ir.g, ana these models can be used in con Jun e t ion/ vl t h the methods 

of ccntrcl theory to derive precise algorithms for sequencing instruc- 

tier, ar.oti^ vccabulary .items. Although our original interest iir this ' 

topic va^prlsiarily th^ol\;tical, the work has proved to have significant 

practical applic^rtlons. These applications Involve computerized vocab- 

, 'olary learning programs designed to supplement college-level courses In 

second-language instruction. A particularly interesting effort involves 

a aupp^leaentary Russlair program in use at Stanford University. Students 

ai« exposed to approximately 1,000 words per academic quarter u&lng the 

mptiter; In conjunction with noroal classroom work this prograr^ enables 
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then to dc'/elcp a ;:ubstantial vocabulary-. ..Many foreigr language In- ' 
structors believe that the major «jt)stacle/t9 *Uicces5ful Instruction in 

a second language l;.^ : ot learning^ th«T gran^^r of tue lat^age, but rather 

In acq^llrirlg a' : ufftclcnt v^ cabuiary that ':hf tuient c«n eiijgage in 

sleanlngful ccnversatioriii and read materials othe^^ than the textbook. 

"In examining the vork on vocabulary acquisjition I will not describe 

tne CAI programs, but will re\^ev some z^seaSrch' on optimal sequencing' 4 

« 

ACtiicz that prcviaes the theoretical rationale tor tne programed It ^ 
will be useVul to describe one experiment in 30me detail before con- 
wtderlr^ more general issues^ 
An Experiment on Optimal Sequencing Schemes 

In thij :.;tudy a large &eX^ of Geiiaan-English items are to be learned 
luring an instructional i^ession that involveti a series of trials* On 
e9:r. trial, one cf |^e German words is presented and student attempts 
tc. *;i/e ::.e £r*giii>n t I'Hnslation; rr.t: correct translation is then pre- 
sentei i\:r a brief study period. A predetermined number of trials is 

i 

allocated for'tht? ln^>t met tonal session, and aftelr some intervening 
pt-riod a te::t is administered over the entire vocabulary. The problem' 
is to ^pecitV * strategy for presenting i^ems during the inatnictional * 

sesi^lcr. JO that performance on tne delayed test will be mfiiximized. 



These CAT vocatulaiy programs fflaKCuse of oj tiaal sequencing achea 
cf the sort to be li&cussed In thlF section, as well as certala mn^nonic 
alis. For a <liscru3r>ion ot '.he^e 1hr.es:onic alJs see fiaugh and AtKln£\n 
^^-i A'^klnscr: anJ Raugn (1975). 
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Four strategies for 3equencin|; the instructional material will, be 
considered. One strategy, designed RO for random order, is to cycle 
tfer^mgh the set cf Items randomly; this strate^ is not expected tc t>e 
^rtlcularly effectiw, but it pnj\rideG a berichniark afflainst whlc^. tc 
evaluate other procedures. A second strategy, designated SSfor self 
selection, la to let the student determine for himself how best to 
sequence the material. In this mode, the student ticcldes on each trial 
which item is* to be presented; the learner rather than an eattema^ co»- 
trcller determines the sequence, of instruction. ^ 

The, third and fourth schemes are based on aj decision-theoretic 
analysis of the task. A mathematical model that providea an accurate 
accour.t of vocabulary acquisition is asstuaed to hold In the present 
^i^xiM^i-n. The model is used to ctsmpute, on a trlal-by-trlal baala, an 
in ll'/l i^-ial f^tud^ntj> curren^ state of learning. Based on these^ccmpu- 
tari/:nE, items are selected ftor test and i;tudy so aa to optimize the 
le'/el cf learning achieved at rte termination of the Instructional ses- . 
ilc:.. Two cptlntization strategies derived from this type of analysis 
will be examined. In o^ case, the computations for detcralnlng an 
'Optimal strategy are carried out assuming that all vocabulary Items are 
of equal difficulty; this str^itegy is designated OE (i.e., optimal under 
the assumption of equal Item difficulty). Ip the other case, the compu- 
tations take into account variations in difficulty levejTamong Items; 
this strategy la called OU (i.e., optimal under the assumption of unequal 
item ai'fficulty). Tlie details cf these two strategics will be described 
later. * ' . 
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Tpt experiment wa;; carried out ur.ier cvTciputer ccntrol : th-p. detail? 
cf the expertaiental pr^^ced^re are glvtrVi In Atkinson il9J2h), The stiirtents 

partlclpatea ir tvc ^^^^ion^: aii "in^: tmcttvnal ^^ej^sidn'' of apprcxiffiiately 
twc :;curs ani a brievfer *Vielayt 5-*^^^: * , r\ r// n ir:tnirteT\ { . r #; uv^k* later 
1*:e lelayei re^t vas the sarne r'cr alX students and Ir.volved a t^t over 
tne entire vocabulary* The Instmctionai sei^i^icn was mere complicated. 
The vocabulary Item;-?, w»re divtded lr;r ^leven lists, each containing 12 
Ci*^rt2«r. ri^ ; t:;e :Lfc:ven liats vt^rt^ arnsmged in a rcunS- robin order* On 

* 2 

each trial T t^:e Iristructicral ;ieti4;lcrs a list we^ displayed on a pro-- 

j^'ctlcn -creer , ar.d the -tudciit l-iSpectfed It for n brief period of ^Jjne ; 

'::e ii;it l:;vclwi only the 12 Gensafi wordt? and net their English trans- 

iatlcr*^. Tnen or.e of tne itenL:^ cn the 'list was selected for test and 

■# - 

' -iy* Ir; tne r^U^ C£^ and uCJ condition^^ the/item was selejcted by the 
r.r.puter: tr. tr.p S3 condirl^jn the ttesi was chosen by the student. After 

Dy *ypir4? !■ : r. i;!^^ coir^p^^ter Cwn;;cle ; then feedback regarding the 
z trvz"^ translation was giwn* The next trial began with the computer 

ir;:piaylr»g r;^xt ilr:t Ir^ tn^ round nhlrip and the saine procedure wa^s 
rept^atel- In^s truct lonax set^sion continued In this fashion for 336 

^.rlai:^ . 

* 

The rei>ult3 of the cxperlir^nt are summarized' in Figure 5# Data are 
prf^sentei on the left side of tr^e figure for perfortnance on ^jucceoslve 
block* f trials dujrtYig the i natructlonal ^esaion; on tfie right are 
r^i^ultri from the test cession administered one week after the InBtruc- 

*I.cnal .-;ei~-sion» T:ie "iata i'roni iri;: r, ricti*>n%l i^er^r^ion are pre;>er;te<i • 

in 3uecej3ive blocks of da trial:;>; ful the FO condition thi£» rfsear;.> that 



on the average each item was presented -©ace in each of these blocks. 

Note that perfonaance during the instructional session Is best for the 

HO. condition, nex^^^^^r the OE eoraitlon which is slightly better 

^^^^^^^^ 

than the SS condition, and poorest for the OU condition. The order of 
the groups is re-/ersed on the delayed test. (iVo points are displayed 
in the figure for the delayed test to indicate, t^at the test involved 
twc randod cycles through the entire vocabulary,; \hovever, the values 
given are me average over the two test cycles.) ^The OU condition is 
beot -.-itr. a correct response probability of .79;, the SS condition Is 



r.ext witn ^53; tne OE ec/ndition follows closely and the RD con- 

/ 

ilticn is poorest at .38. The observed pattern of results is J#hat one 
wculd expect. In the SS condition, the students are trying to test 
\:.e-sel\'ef^^t: xtemi5 the-y do not knowj consequently, during the instruc- 
ti^:;ai ::ecGlcn, they should h^ve a love^proportion of correct responses 

*r.a:. .5"u:er.ns run or. tne RO p re ce dure/ where items are tested at random. 
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_^irJ.larly^ tne OE and CU ccnditicns 3?x5volve a procedure that attempts to 

lientiiV and test those items \hat have not yet been mastered and should 

produce high error rates during t^ instructional session. The ordering 

groups on the delayed test is reversed since all words are tested in 

a non-selective fashion; under these conditions the proportion of correct 

responses provides a measure of a student's true mastery of the total 

set of vocabulary items, ^ 

. , . # ' , — ^^jk 

■ The mfi^nitude or^^ effects observed on the delayed ft ^t are of 

! 

prastical significance. The SS .condition (when .qocj^red to the fiO 

cf^TKiition) le*ds t<^.^_yelative gain of 53^, whereas the OU condition 



yields.^ ^ relative gain of lOdf). It is i^terestir^g that students were 

ho 



i^iter 



somewhat effective in determining an . ptimal study sequence, but not so 

efftetive aa the best of the two adaptive teaching systems, 

Rationale for Sequencing Schemes 

Both the CU and OE schemes assume that vocabulary learning can be 
f 

described by a fairly simple model. We postulate that a given item is 

in one. of three states (P, T, and U) at any moment in time. If the item 
is in State P, then its translation is known and this knowlldge is 
"relatively" pemanent in the sense that the learning of other items 
will not interfere with it. If the item is in State T, then It is also 
kno'i^ but on a "temporary" basis; in State T the learning of other items 
can give rise to interference effects that cause the item to be foigotten. 
In S^^ate U the item Is not known, and the' student is unable to give a 

I 

trar.;-. latton. 

• ■ Wr.er. Item i is presented on a .trial during^the instructional session, 

the fcllcwir.^ transition matflx describes the" possible chance in Its state 

P T U 

' Pf 1 0 0 

L(l) = T x{i) l-x(l) 0 

' uLy(i) zdr l-y(i)-z(i) 

Rcw5 of . the matrix represent the 3ta«e of "the item at the start of the 

> 

trial, and columns th^ atate at the end of the trial. On a trial when 

■ some item other than Item i is presented for test and study, ..transitions 
in 'the'state^ Item i also may take place. Such transitions can occur 
only if the student makes an error to the. other item; in that*case the 
transition matrix applied to Item 1 is as follows: ' . • *. 




F(i5 - tJ ^ i.f<i);/ f(i) 

uio 0/ 1 J 



/' 

Ba&icall/, the ijiea Is that when sooe/'ot^ier item is presented that the 

' ' , '' / 

Student does not Know, forgetting my occuy for Item i If It Is In 
State ^ / ' 

To siuamfllze, when Item 1 Is presei^ted for test a»d study ^ transi- 
tion Matrix L(i) Is applied; when soioe/other item is pKsented that 

■ y ■ ' ■ ■ 

elicits fit\ error, MatrU F(i) is applied. It Is also assumed that at 

the start of the instructional ses^on Item i is either in State vlth 

/ 

probability g(i), or in State U, ^th probability l-g(i); the student 
either knows the translation wit|((out having studied the item or does 

not. The above ass provide a ccngplete description of the learning 

/ 

process. The parameter vector/{x(l) , y(l), z(i), f(l), g(l)l charac- 

/ 

terlzes the learning of Item ^ in the vocabulary set* The first three 

/ 

parameters govern the .acquisition process; the next parameter^ forgetting; 

/ * 

and the last, the student •s Knowledge prior to entering the experiiaent* 

We now turn to .a discussion of how the OE and OU proced^tree were 
derived frcan the model* Pl4.or to conducting the e3q)eriment reported 
here, a pilot study was xun using the same word lists and the RO pro- 
cedure described above^ ^Data from the pilot study wei« employed to ^ 
estimate the parameters of the model; the estimates yiere obtalniM using 

fflinifflUBi chl-square procediires descitLbed in AtkinBcn (1972b). Two 
separate estimates of paresneters were oade. In one case it was assiuoed 
that the iteias were all equally iifficxilt, and data from all 8i» items 
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were luaped ^together to obtain a single estl-r.ate of tht* parameter vector 

this estimation procedure v±ll be called the equal-parameter case (E 

'.■ST 

case)* In th^' eecond case the data were \>eparated tiy Iteme^ and an e^l 

* 

2:ate of the parameter */ecr< r wa: rr^aje fur each » f ^r^e 8^ Item, i thiit 
procedure vill called the ^x^equal*paralneter case (U case). The two 
sets cf paraine^er e^tlr;ates were then xised to generate the optimization 
cchesie;; previously referred tc as the OE and OU procedures. 

In crdtT to rcroulate an instructional strategy^ it s necessary to 
be/TU*j5>*#-»tcut t.ne q-iantity to be. maxinilzed. For the present expert- 



mer.* "nt* goal is to max i mi re tne *otal iuir.be r of Items the student 

5 

cwrr^ „*ly tran^iaLe? on t;*e delayed test. To do this, we need to 
apt-clrV r^laticnship be*vaen'^he state of learning at the end of 

ir -r^ctional secssi n and performance on the delayed teat. The 
a. t:,:;. Kade r.^re it that only those item^ in State P at the end of 
* t 1:.. • p^c* : r.al : will oe traniiated correctly on the delayed 
*.r ♦ , a: itprr In ST ate T is prejunied t o be forgotten during the inter- 
*^t:.f s ,'i't*k, T^^Ji^/ rhf prcblens of maximizing delayed- test perfoimance 
i,-.v^*vi". r.ax^.ii.'.irig *.ht- number of ittrsii in State P at the end of the 

r., 



■^ ^-^ner iKeateur* can be used to assess the benefits of an instructional 
strategyi ^f.g., in thia case weights could be assigned to items measur- 
ing their Wlati-'/e iniportance. Also costs may be afcao'cl«t«;d with the „ 
vanout actionr) tak^-n during an instr-iotional session* Thus, f<5r the 
general cajf , tnr op'tsiiyat'tcr, probl5*m involves assessing costs ar«d 
6^npf:t> and finding a strategy that naxiittises an appropriate function 
ieftnej -..n Fcr a ii: cut^cion • f thest pcintf; see Dear, Silberman, 

Estavan, and Mkin-on (I967), and Smallwood {1962, 1971)- \ * 
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Havlpg nuffle^cal valuer for parameters and Knowing a etuderrt's 
response history, it Is possible to estimate his current state of learn- 
ing,^ Stated mere preclsely*> the learning model can be i^ed to derive 
equations and, in turn, compute the probabilities of being in Ctates P, 
T, and U for each item at the start of any trial, conditlonalized on the 
student* 3 response history up tc that trial. Given numerical estimates 
of these probabilities, a strategy for optimizing perfomance is to 
select that item for presentation that has the greatest probability of 

moving into Sta*" ?• This strategy has been tenned the one*stage opti- 

/ 

aiization procedure because it locks ahead one trial in making decisions* 
The true cptlaal policy (i.e., an N*stage procedure) vould consider all 
possible item- response seqxiences for the remaining trial^s and select the 
next item so as to maximize the number of items in State P at the ter- 
niinaticn of the instxMctional session. ^ Unfortunately, for the present 
case the N-stage policy cannot be applied because the computations are 
tec time consuming even for a large computer. Monte Carlo studies 
indicate that the one-stage policy is ja good aj^roxlmatlon to the , 
cptlaal strategy; it was for this reason, as well as the relative ease 



The student's "response history'* is a record for each trial of the 
vocabulary item presented and the response that occurred. It can be 
shown that there exists a "sufficient history" that contains only the 
inforaatlon necessary to estimate the student •s current state of 
learning; the sufficient hlstoiy is a function ot the CMiplete history 
and the aasu^d learning model (Groen & Atkinson, I966}. For the model 
considered liT this paper the sufficient hlstc^ Is fairly single. It 
Is specified in terns of individual vocabulary items for each student; 
we need to know the ordered sequence of correct and incorrect responses 
to a given item plus the number of errors {to other items) that inter- 
vene between each presentation of the It^^fli. 



of computing, that the one-stage procedui* wae eaployed. For a discus- 
sion of one-stage and ll-etage policies and Monte Carlo studies comparing 
thea see Groen and Atkir.^vn (I966), Oalfe« (1970), and Laubsch (1970). 

The optiaiizaticr. pn>cea\*rfr ..^■'i-crH-<-<i at^-.'/e vBu iRiplementf-i.cn the 
computer and permitted decislonis, to be made Tor each student on .a trial- 

by- trial basis. fS'S-^udents in th$ CE group, the computationa were^ 

\ 

carried cuu.usirig the five parameter values estimated under the aasump- 
ticn of .nomc^eneou.- itemfi (E case); for aVdents in the OU group the 
/ ccaputatlcns v^re based or. the h20 parameter V^alues estlnated under the 
assuT-ptlor. cf neterogen-^otid itemi (U case). 

The OU prcct»dure is sensitive to Interlteai differences and conse- 
i^uer.tlv ger.erates a more effective optimization strategy than the OE 
pTccelur€>. The OE procedure, hcwewr, i3 almost as effective as having 
tr.e^iudent nake r.ls own Instructional declsionis and far superior to a 
. r*i::.i ./■•r. prejenTat icn scneisie. 

The f^tudy reported here is one In a series of experiments dealing 
with cptL'sal ^>equencing :^:;heme3. Ir was selected because it is 'easily 
. dejcrioe i a:;3 p*?^nnits direct compdrli>on between a learner-controlled 
procedure y^r^5lJts procedures based on a decision- theoretic analysis* For 
a review of otner studies similar tc the one reported above see Chiang 
(I91k), Deiariey (197^), Laubsch (1970), Kimball (1973), Paulson (ITJl) , 
and AtKlnson and Paulson (1972.)^ Some of these studies examine, pro- 
cedure;^ that are aore powerful than tne one:^ described hens, but they 
are complicated and difficult to describe vithcut going Into mathematical 
detail. The major -'iropro'/snientii invclvc two factors : (1) methods for ■ 

* 

estimating the model's parameters during th^ course of instruction, and 

* 
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(2) oore sophisticated j>ays of interpreting the model^s parapieters to 
take account- of both differences among students and differences among 
items. For cxBmple ), let P{i,j) be a generic symbol for a peirameter 
vector characterizing student I learning vocabulary Item J. In these 
studies P(i,j) is specified as a funct'.on of a vector A(l) measuring the 
ability of student i and a vector D(J) measuring the difficulty of item 
J.. The prc:bl€!C then is to estimate the ability level of each student 
and the difficulty of each item while tne student is running on the 
prcgram. In a study rep<:;rted in Minson and Paulson (1972), rather 
iramatic re^ulta -wereNsbtained using such a pr.5cedure. A special feature 
cf th". study was that students were run in successive groups, each 
starting after tne prior group had completed the experiment. As would 
be expected, the overall gains increased from one group to the next.- 
The reason is that for the first group of Students the estimates of item 
'iirricul^y^ ^^r?^ :rui^' tut improved with the accumulation of data 

from each successive vave of estudents. Near the end of the study e3ti* 
ciates of D(J) were quite precise and were essentially constants In the 
^yr^tem* Th«e only tasR that remained when a new student came on the 
system was to estimate A{i); truit is, the parameters characterizing his 
particular ability .level* This ^^-udy provides an example of an adaptive 
Instructional system that meets both of the requirements stated earlier 
in tnis paper. The sequencing of instruction varies as a function of 
each -student *a history re^cord, and over time the system Improved in 
efficiency using data froiB prevlouE^ students to sharpen Its estimates 
of the difflciilty of irnstrjct lonal materials* 



cxaicmnnic RatAms 

The projects described In this paper have- one thene in comndn, 
n«ly, developing cofl^ter-controUed procedures for optimizing the 
instructional process. For several of the instructional tasks consid- 
ered here, mathematical models of the learning process were fomulated 
which made it -possible to use formal methods in desiving optimal policies. 
In otiier cases the "optimal schjemfcs" vere not optimal in a well-defined 
sense/ but were based on our intuitions about learning and some relevant 
experiments. In a sense, the diversity represented in these examples 
corresponds to the state of the art in the field of instructional design. 
For some tasks we can use psychological theory to help define optimal 
procedures; for others our intuitions, modified by experiments, must 
guide the effort. Hopefully, our understanding of these matters will 
increase as more projects are undertaken to develop sophisticated in- 
structional "^procedures. , 

Some have w^ued that any attempt to devise optimal strategies is 
doomed to failure, and that the learner himself is the best Judge of 
appropriate instructional actions. I am not sympathetic to a learner- 
controlled approach to instruction, because I believe its advocates are 
trying to avoid the difficult but challenging task of developing a viable 
theory of instruction. There obviously is a place for the learner's 
Judffnents in making instructional decisions; for example, such Judg- 

i 

ments play an U^rtant role in several parts of our BEP course. However 
U4lng the learner's Judffnent as one of several items of information in 
making instructional decisions is different from proposing that the 
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learner should nave coaji^ete control'. Results presented in this paper 

•fid those cited in Beard, \x>rt on, Searle, and Atli^Jison (1973) Indicate 

that the learner is not a pa^icularly effective decision maker In 

guiding the learning process. \ ( 

Elsewhere I have defined th^ criteria that must be satisfied before 

t 

an optimal instructional procedure "^an be derLvect using foimal methods 
i 

(AtHinson, i972a). Poughly stated, they require that the .following 

' < \ 

elements of an ^^nstrjicticnal situation be clearly specified: 

\ 

(1) The* set of admissible lnstructi<5>nal act'lons 

(2) The instructional objectives 

(3) A measurement scale that pemits costs to be assigned to 
each of the instructional actions '^and payoffs to the achieve- 
ment of instructional objectives ^ 

(i^) A model of the learning process ^ 

If t.nese four elener.ts can be given a precise Interpretation, then It Is 

ft- 

usually pcjjitle lo ieri-/^j ar. op:ln;al ir.^ fractional policy. The oclution 
for an optimal policy i^ not guaranteed^ but in recent years pcverful 

/ toola have been developed for discovering optimal, or near optimal, pro- 

* 

cedures if they exis** I will no^ dl!?'cuss these four elements here except 
« to note that the first '^ree can usually be specified with a fair degree 
on consensus. Issues of short-term versus long-tenn aesessnients of costs 
and payoffs raisfe important questions r?3garding educational policy, but 
at least for the types of instructional situations examined in this paper 
reasonable specifications can be offered for the first lyve elements. 
However, the fourth ele3»nt"the specification of a model of the learning 
proces3*-represent5 ajnajor obstacle* Our theoretical understanding of 
learning Is so limited that only in very special cases can a model be 



specified In enough detail to enable the derivation of optimal procedures 
Utotll ne haw a much deepf r understanding of the-leamlng^ procees, the 
identification of tnOy effective strategies will not be possible. Sow* 
ever, an all-inclusive theoiy of learning is not a prerequtfllte for the 
dewlopoent of optimal proceduyes. What is needed is a model that 
eaptuces the essential features of that part' of the, leem^ng process 
being tapped by a given instructlonal^^ssk. Even models that have been 
rejected on the basis of laboratorjr investigations msy be useful in. 
deriving instructional strategies. Several of the learning models con- 
8idex«d in this paper have proven unsatisfactory when tested In the 
laboratoiy and evalxiated using standard goodness-of-fit criteria^ never- 
theless, the optimal strategies they generate are often quite effective. 
My own preference is to foimulate as complete a learning model as 
intuition and da%a will peimit and then use that model to investigate 
optlaal procedures. When possible the learning model should be repre- 
sented In the form of mathematical equations,* but othfrwise as a set of 
statements .in a cooiputer-Blmul^tlon program. The main point is that the 
development of a theory of instruction c%mot progress if one holds the 
view that a comprehensive theory of gaming is a prerequisite. Rather, 
advances in learning theory will affect the development of a theory of 
instruction, and, conversely the development 'of a theory of instruction 
will ixLCiWRce thi direction of research on learning. ■ * \^ 
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